Face Pose Estimation in Uncontrolled Environments

Automatic estimation of head pose from a face image is a sub-problem of human face analysis with widespread applications such as gaze direction detection, human computer interaction or video teleconferencing. It can also be integrated in a multi-view face detection and recognition system. Current methods on face pose estimation from a 2D image can be divided into two groups. The first group : geometric shape or template based methods (e.g. [3]) use a set of landmarks such as the relative position of the eyes, mouth, etc. or a template such as an Active Shape Model (ASM) to estimate pose. The second group: manifold learning methods (e.g. [1]) use linear/non-linear embedding methods to learn a lower dimensional space in which they estimate pose. One limitation of current methods is that most of them estimate pose in a limited range and treat pose estimation as a classification problem by assigning the face to one of many discrete poses [3]. However pose estimation is truly a regression problem. Another drawback of current methods is that they have mainly been tested on faces taken in controlled environments i.e. with solid background and small or no variation in illumination and expression. In this paper we propose a probabilistic framework for continuous pose estimation. We use a general image representation that does not rely on locating facial features. This representation is inspired by recent successes of patch-based methods which have shown to be highly effective for other areas of computer vision such as texture generation [2]. We use this representation in a generative model for automatic estimation of head pose in “real world” images ranging from −90◦ to 90◦. Our approach breaks the test image into a non-overlapping regular grid of patches. Each is treated separately and provides independent information about the true pose. There is also a predefined library of object instances. The library can be considered as a palette from which image patches can be taken. We exploit the relationship between the patches in the test image and the patches in the library to estimate the face pose. In inference (see Figure 2), the test image patch is approximated by a patch from the library L (Fig. 2b,c). The particular library patch chosen can be thought of as having a different affinity with each pose. These affinities were learned during a training period and are embodied in a set of parameters W (Fig.2d). The relative affinity of the chosen library patch for each pose is used to determine a posterior probability over pose (Fig. 2e).